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Abstract. We propose a maximum entropy (ME) based approach to smooth noise not only in 
data but also to noise amplified by second order derivative calculation of the data especially for 
electroencephalography (EEG) studies. The approach includes two steps, applying method of ME to 
generate a family of filters and minimizing noise variance after applying these filters on data selects 
the preferred one within the family. We examine performance of the ME filter through frequency and 
noise variance analysis and compare it with other well known filters developed in the EEG studies. 
The results show the ME filters to outperform others. Although we only demonstrate a filter design 
especially for second order derivative of EEG data, these studies still shed an informatic approach 
of systematically designing a filter for specific purposes. 
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INTRODUCTION 

Filtering out high frequency noise becomes extremely crucial in source localization 
problem of invasive EEG studies. Based on Freeman and Nicholson's works, source 
localization requires calculation of second order derivative of the EEG data [1, 2], Be- 
cause the derivative amplifies especially high frequency noise, Freeman and Nicholson 
proposed to introduce low-pass non-recursive filters to smooth it. [2]. Suppose a set of 
discrete data, 4> (r) labeled by spatial position r, which are distorted by noises £ r and 
these data are equally spaced sampled with interval h. The simplest non-recursive filter 
is then defined by simply averaging data at r and its neighboring data with specific set 
of weighting values pi, 

4>,(r)= £ Pl <t>{r + lh) , (1) 

l=-z 

where z is an integer and 2z+ 1 data points are considered. We will designate this 2z+ 1- 
point spatial filter as a S( 2z +i) filter. The mean filter, in which pi equals to l/(2z+l), is 
one filter widely used in many fields for data smoothing. Its applicability, however, is 
sometimes limited to data types [3]. 

Many methods have been developed to design low-pass filters in many fields. For ex- 
ample, there are adaptive beam-forming ([4] - [7]), wavelet de-noising [8], and entropic 
method ([9] - [14]) etc.. Yet no explicit discussions are given on smoothing noise in 
second order derivative of contaminated EEG data since Freeman and Nicholson's pio- 
neering works. Freeman and Nicholson's approach will be briefly discussed in Sec. 2. 



Two drawbacks, requirements of some empirical guidelines and low accuracy of second 
order derivative, in their approach will also be addressed. 

Our goal is to provide a ME based approach to filter out high frequency noise espe- 
cially in second order derivative of contaminated EEG data. This approach eliminates 
the two drawbacks in Freeman and Nicholson's approach and provides a systematic and 
robust method for smoothing noise. The ME approach and its application in the EEG 
studies will be presented in Sec. 3 and 4 respectively. We thereafter examine the pro- 
posed filter's performance and compare to some other filters such as the mean filter, 
Freeman and Nicholson's [2], and Rappelsberger's [15] filters through frequency and 
noise variance analysis in Sec. 5. A conclusion is given at last. 

SOURCE LOCALIZATION FROM THE INVASIVE EEG DATA 

In neurophysiology, electric field potentials are initiated during neural activity when 
ions flow across localized regions of cell membranes. These ionic flows establish a 
distribution of current sources and sinks in the extracellular space [1, 2, 16]. These 
field potentials can be measured in extracellular space discretely by the invasive EEG 
instrument, which usually has 16 to 32 detectors on its probe. 

According to theory of electromagnetism, the current sources and sinks are associated 
with second order derivative of the field potentials. Freeman and Nicholson proposed the 
current source density (CSD) method for source localization [1,2]. The CSD computes 
second order derivative approximately through substituting specific smoothed <& s (r), 
Eq. (1), into Ta3(r) = o c /h 2 (4>(r — h) — 24>(r) +<&(r + h)), where three data are con- 
sidered, G c is conductivity, and h is separating distance between two sensors. A final 
expression of an approximated second order derivative is given by 

M 

D 2 (r) = o c /h 2 £ a m <S>(r + mh), (2) 

m——M 

from 2M+1 datum with weightings a± M = Pz> a ±M^i = Pz-i ~ 2p z , and a± M ^k = 
p z -k — 2-Pz-k+i + Pz-k+2, in which k > 2. The preferred smoothing function should be 
able to reduce high frequency noise and noise amplified by operation of the Ta3(r). 
Freeman and Nicholson devised several smoothing functions as shown in Table. 1, 
denoted by FNSn ([2]), where Sn stands for n-point spatial smoothing. Notes that the 
FNS3(2) is still used in Karwoski et al. [17] and Ulbert et al. [18]. These pi are derived 
from the least square method [19, 20]. In addition, they also introduce some ad hoc 
rules to obtain 5 and 7-point filters. For example, the FNS5 is obtained by smoothing 
data twice using set of (3/10, 4/10 ,3/10). Later, several successors have designed other 
smoothing functions according to their needs such as the Rappelsberger's RS3 filter [15]. 

There are two drawbacks in Freeman and Nicholson's CSD method. The first is re- 
quirements of some ad hoc empirical guidelines for designing smoothing functions such 
as FNS3(1) or RS3 shown in Table. 1. These empirical guidelines may limit the use 
of these smoothing functions to some specific EEG data. For example, Freeman and 
Nicholson's smoothing functions are suited for Anuran cerebellum studies [2]. The sec- 
ond is these smoothing functions are devised for second order derivative calculated via 



Ta3(r), which is not an accurate calculation. In next section, we propose an informatic 
approach based on the ME method to eliminate these two drawbacks. 

TABLE 1. Filters in EEG studies. The 
table lists five smoothing functions [2, 15]. 
MF: mean filter, FN: Freeman and Nichol- 
son, R: Rappelsberger. 





PO 


P±\ 


P±2 


P±3 


MFS 3 


0.33 


0.33 








FNS 3 (1) 


0.5 


0.25 








FNS 3 (2) 


0.43 


0.29 








RS 3 


0.54 


0.23 








FNS5 


0.34 


0.24 


0.09 





FNS7 


0.26 


0.21 


0.12 


0.04 



A MAXIMUM ENTROPY LOW-PASS FILTER 

Rationale. The method of ME is developed as a tool for assigning a probability distri- 
bution of observing a system to be at a certain state according to information in hand, 
which is in the form of constraints [21]. Based on conventional studies on properties 
of the noise filter, Eq. (1), in the EEG source localization problems, which will be il- 
lustrated later, one can treat the weightings pi as a probability distribution. Thus the 
ME provides a robust and objective approach for determining a preferred set of p { once 
constraints relevant to noise reduction in the EEG problems are given. 
The constraints. Conventional studies in EEG data analysis have shown some properties 
that are relevant to smooth noise using Eq. (1). First, the p t are positive decimal values 
and symmetric around I = 0. These values obey a normalization condition Y^=- z Pl = 1- 
Second, the preferred set of the pi should have noise in data being mostly removed 
through the 4\ s (r). We expand right hand side of Eq.(l) with respect to r according 
to the Taylor expansion and use Y?i=- z Pl = 1 an d Pi = P-i, which results in the odd 
order derivative terms in the expansion vanished. The final result is 4> s (r) = 4> (r) + 

4>( 2 ) (r) h 2 &i/2-\ , where 4>( 2 ) (r) denotes exact second order derivative of clean data 

and so on and 82 = L/ = _ z /V 2 - It indicates that smoothing ability is actually related to the 
82 value. Summarizing the above results, suggests the weightings pi to be a probability 
distribution of having the Eq.(l) to reduce noise by the 82 approximately. Thus the ME 
can be applied to determine the preferred set of p t . 

The preferred low-pass filter for data smoothing. The ME states that maximizing 
entropy, S = — Y%=- z Pl m Ph subject to the two constraints, the normalization condition 
of pi and 82 = Y, Z i = - z Pil 2 gives a preferred form of the pi, 

p z =Z- 1 exp(-a/ 2 ), (3) 

where a is a Lagrangian multiplier and partition function Z = E/=_ z ex P( — Oil 1 ). The 
preferred set of pi is a function of a. Namely, the ME method generates a family 
of pi (a) distribution. In principle, the Lagrangian multiplier a can be determined by 
substituting Eq. (3) back into &i = Y, Z i = - z Pll 2 ' which gives 82 = —Z~ 1 dZ/da, if the 82 



value is given. Unfortunately, the &i value is unknown at this point. Thus, we propose 
an alternative method to determine the a. 

Since the preferred a value should result in corresponding set of the pi to mostly re- 
duce noise, if one can quantify noise the preferred a value can be determined. Although 
it is difficult to quantify noise, noise variance can be quantified by the autocorrelation 
function [3, 6]. Suppose field potentials 4> (r) = 4>o (r) + £ r is repeatedly measured sev- 
eral times, in which a Gaussian white noise £ r with variance o is considered. For white 
noise, we have (e r ) = 0. Besides, noises are uncorrected at different data points r and 
r', (e r £ r i) = o 2 for r = r' and for r ^ r' [3]. Because noise has zero mean, mean 
($0 ( r )) is identical to 4>o (r), noise-free data. Thus one can find autocorrelation of <J> (r) 

is C a = ^(4>(r) - (4>(r))) 2 ^ = (^(e r — (£ r )) 2 ^ = <* 2 , the noise variance. Similarly, one 

can compute the autocorrelation function of 4> s (r), C smoo th = (^(<&s ( r ) — (®s ( r ))) 2 ^ = 

G 2 Y?i=- z P 2 '■ It indicates noise variance being amplified by E; = _ z p 2 ar ter data is pro- 
cessed by the Eq.(l). Thus the preferred a value should minimize the noise variance 
Qmooth, which is found to be zero and p t (a = 0) = l/(2z+ 1). This is exactly the mean 
filter obtained from the least square method [3, 19]. Yet the ME approach roots in re- 
ducing noise variance C smoo th mostly, which is not clear in the least square method. 



FILTER OUT NOISE IN SOURCE LOCALIZATION 

Basic features. Freeman and Nicholson suggest to identify current sources and sinks 
through calculating second order derivative of the field potential via the Ta3(r) with 
specific smoothed 4>. v (r). However, the Ta3(r) does not provide accurate calculation. 
One straightforward way to raise the accuracy is to incorporate more data points into for 
calculation, 

M 

Ta 2 M+i(>-)= £ al<f>(r + mh), (4) 

m——M 

where 2M+1 data points are considered. To determine coefficients one first 

expand <&(r + mh) with respect to r based on the Taylor expansion, Ta2M+l (r) = 

^ = _ M a^(r) + l^ = _ M almh^Hr) + l^ = _ M aUmhf By re- 

questing coefficient of second order term, Em=-M a m ( m ^) 2 /2 = 1, and rests are zero, 
we have approximated Ta2M+i ( r ) to be identical to exact second order derivative 
4>( 2 ) (r). One then can solve for coefficients given these criteria. For three point 
calculation, coefficients are given in the Ta3. Coefficients a\ 2 = — 1/12, a±j = 16/12, 
and a T Q = -30/12 for Ta 5 and a\ 3 = 1/90, a T ±2 = -13.5/90, a T ±x = 135/90, and 
a T Q = -245/90 for Ta 7 . 

Given these coefficients, one can immediately find the noise variance to be amplified 

by C D2 = (ip 2 (r) - (D 2 (r))) 2 ^ = o 2 I^=-Af («m) 2 aft er operation of the Ta 2M +i (r). 

For example, it is 6, 8.03, 9.75 times of the original noise variance for using Ta3 (r), 
Tas (r), and Ta 7 (r) respectively. The noise amplification is proportional to the number of 
data utilized in the calculation. This result suggests a trade-off of raising the accuracy of 



TABLE 2. The preferred ME filters, denoted by MES n Ta, 





C Ds2 (xa 2 ) 


I a 


P0 


P±\ 


P±2 


P±3 


MES 3 Ta 3 


0.28 


| 0.031 


0.42 


0.28 








MES 3 Ta 5 


0.38 


| 0.033 


0.43 


0.28 








MES 3 Ta 7 


0.42 


| 0.035 


0.44 


0.27 








MES 5 Ta 3 


0.048 


| 0.017 


0.29 


0.23 


0.12 





MES 5 Ta 5 


0.061 


| 0.018 


0.29 


0.23 


0.11 





MES 5 Ta7 


0.065 


| 0.019 


0.30 


0.23 


0.11 





MES 7 Ta 3 


0.014 


| 0.011 


0.22 


0.19 


0.12 


0.06 


MES 7 Ta 5 


0.017 


| 0.012 


0.23 


0.2 


0.12 


0.05 


MES 7 Ta 7 


0.019 


| 0.012 


0.23 


0.2 


0.12 


0.05 



second order derivative calculation with more data is amplification of the noise variance. 
Thus it is unlikely that the Freeman and Nicholson's and others filters can smooth this 
noise amplification because those filters are designed for Ta3(r) calculation only. One 
needs new smoothing functions. 

The preferred ME filter. The new smoothing functions can be obtained through the 
following procedure. Substituting smoothed data, Eq. (1) into second order derivative 
Ta2M+i (r) first gives a generic expression 

M+z 

D s2 (r)= £ a' n ^(r + nh), (5) 

n=-(M+z) 

where a' n is a function of and pi. The preferred set of pi is then determined by mini- 
mizing variance C Ds2 = ((D s2 (r) - (D s2 (r))) 2 ^ = <7 2 L^ (M+z) < 2 . It should have high 

accuracy of second order derivative calculation and minimum noise variance simultane- 
ously. Table 2 gives the results. We define operation based on the ME that utilizes n 
data points for noise smoothing, S n , and m data for calculating second order derivative, 
Ta m , as MES n Ta m in the first column of Table. 2. Second column records the minimum 
Cd s2 value. Next five columns list the corresponding a and the pi value consecutively. 
The table first shows that when there is more data, S3, S5, and S7, to be considered for 
smoothing given Ta m , the noise variance Cd s2 will be decreased dramatically. However, 
when one smoothes data with S n and calculate second order derivative using more num- 
ber of smoothed data points, Ta3, Tas, and Ta7, the noise variance Cd s2 will then be 
amplified as expected. 

PERFORMANCE ANALYSES 

Instead of applying the proposed ME filter directly to study real EEG data for exam- 
inations, we will only analyze the filters's performance through frequency and noise 
variance analysis in this work. In addition, we will compare it to other filters, mean filter 
(MF), Freeman and Nicholson's (FN), Rappelsberger's (R) filters from Table 1. 
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FIGURE 1. Frequency analysis of the five different 3-point S3 filters in panel (a) and different S3,Ta3 
in panel (b). 



Frequency analysis. The frequency analysis applies the transfer function H (00) 
for examining changes of signal's amplitudes after signal being processed by the 
filters and second order derivative calculation [3]. The transfer function H (co) — 
exp(— icor)<i> s (r) = po + 2 Hf = i Picoslco is derived by substituting 4>(r) = exp(zcor), 
which is shown to be an eigenfunction of a linear time-invariant system, into & s (r) of 
the Eq.(5), where angular frequency CO equals to 2nf rotational frequency [3]. When 
the transfer function is less then one, it indicates signal amplitude being attenuated by 
the filter. Panel (a) of Fig. 1 plots the results of using the different filters from Table 
1. This figure demonstrates the mean filter has the largest amplitude attenuation and is 
followed by FNS 3 (2) and MES 3 , FNS 3 (1), and RS 3 consecutively. 

Similarly, one can compute the transfer function of second order derivative of 
smoothed data. Afterward, we compute the ratio of calculated and real second order 
derivative value through R(co) = -D s2 (r)/co 2 = -l/co 2 (ao + a\ cos Ico) for 
explicit comparison of effects of the smoothing. When the ratio R(co) equals to one, 
the calculated second order derivative of the smoothed data is identical to real value. 
Panel (b) of Fig. 1 plots not only the calculation through five filters, but also the 
three second order derivative calculations Ta 3 , Ta 5 , and Ta7 respectively. First of all, 
frequency attenuation is the smallest in Ta7, and is followed by Tas and Ta 3 . That 
explains again, why the second order derivative is more accurate when more data is 
comprehended for calculation. Given the second order derivative calculation with fixed 
number of smoothed data, for example, all S 3 Ta 3 in this figure, it suggests that the mean 
filter MFS 3 Ta 3 again has signal with high frequency attenuated the most among these 
filters. These attenuation patterns are similar to the panel (a), which is not surprising 
because smoothing data first and then calculating second order derivative is identical to 
calculating second order derivative first and then smooth the calculation. 
Noise variance analysis. It seems the mean filter to be superior to all other choices from 
the previous analysis. However, the following noise variance analysis shows different 
results. We substitute the filters from Table 1 into Eq. (4) to compute new coefficient a' m , 
and evaluate noise variance Cd s2 . The results are listed in Table 3. Because we design 
the ME filter to mostly reduce noise variance, the analysis shows the MES 3 Ta 3 filter to 
exactly has the best performance among all filters. 

Besides, Freeman and Nicholson have found noise in field potential measurements 
to be proportional to a K- value, K = Li=-(m+ \ \ a 'i\ PL Thus we calculate the K- value 



TABLE 3. Comparisons of filters. The table lists a' n value and noise variance 
Cd s2 ( xct2 ) m tne third column and K value derived by Freeman and Nicholson 
for noise estimate in the fourth column. 





a' 


a '±\ 


a' ±2 


fl' ±3 


fl' ±4 


C 0j2 (xC7 2 ) 


K 


MFS 3 Ta 3 





-0.333 


0.333 








0.44 


1.33 


FNS 3 Ta 3 (l) 


-0.5 





0.25 








0.38 


1.01 


FNS 3 Ta 3 (2) 


-0.286 


0.143 


0.286 








0.28 


1.13 


RS 3 Ta 3 


-0.623 


0.082 


0.229 








0.57 


1.34 


MES 3 Ta 3 


-0.284 


-0.144 


0.286 








0.28 


1.14 




FNS 5 Ta 3 


-0.2 


-0.05 


0.06 


0.09 





0.073 


0.63 


FNS 7 Ta 3 


-0.1 


-0.04 


0.01 


0.04 


0.04 


0.019 


0.39 


MES 5 Ta 3 


-0.116 


-0.055 


-0.007 


0.12 





0.048 


0.48 


MES 7 Ta 3 


-0.06 


-0.038 


0.003 


0.003 


-0.06 


0.014 


0.27 



after using different filters and list in the fourth column of Table 3. The K- value shows 
although MES3Ta 3 is slightly outperformed by the FNS3T 3 (1), which has the same 
performance as the FNS3T3(2), the ME filter is superior than all other filters when more 
data points are included for smoothing. 

Discussions. The frequency analysis has shown the mean filter MFS3 and the MES3 
and FNS3(2) mostly attenuate high frequency part in the signal. Yet the noise variance 
analysis, comparison of the Co, 2 and K value, shows the MES3Ta3 and the FNS3Ta3(2) 
can mostly reduce noise variance among all three-point filters. These three analyses 
suggests the ME filter not only mostly reduce noise variance in second order derivative 
but also has the ability to mostly attenuate high frequency signal. 

CONCLUSIONS 

The filter design delicately depends on one's purpose and properties of a target system. It 
sometimes involves some ad hoc empirical guidelines that suit ones need. For example, 
Freeman and Nicholson have designed several filters for their needs [2]. We propose and 
demonstrate an informatic approach to design a low-pass filter based on the method of 
ME. In addition, we examine and compare the preferred ME filter with other well known 
filters in EEG studies through frequency and noise variance analysis. The performance 
of the preferred ME filter MES3Ta3 and the FNS3Ta3(2) filter are found to be almost 
identical. Even though the ME filter is designed for second order derivative calculation 
especially for the EEG data in this work only, the approach's systematics and flexibility 
ensures one can design filters that suit other purposes robustly without introducing ad 
hoc empirical guidelines. 
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